Improving the quality of machine learning in health applications and clinical research

For machine learning developers, the use of prediction tools in real-world clinical settings can be a distant goal. Recently published guidelines for reporting clinical research that involves machine learning will help connect clinical and computer science communities, and realize the full potential of machine learning tools.

Trends in 28-Day Mortality of Critical Care Patients With Coronavirus Disease 2019 in the United Kingdom: A National Cohort Study, March 2020 to January 2021.

OBJECTIVES: To determine whether the previously described trend of improving mortality in people with coronavirus disease 2019 in critical care during the first wave was maintained, plateaued, or reversed during the second wave in United Kingdom, when B117 became the dominant strain. DESIGN: National retrospective cohort study. SETTING: All English hospital trusts (i.e., groups of hospitals functioning as single operational units), reporting critical care admissions (high dependency unit and ICU) to the Coronavirus Disease 2019 Hospitalization in England Surveillance System. PATIENTS: A total of 49,862 (34,336 high dependency unit and 15,526 ICU) patients admitted between March 1, 2020, and January 31, 2021 (inclusive). INTERVENTIONS: Not applicable. MEASUREMENTS AND MAIN RESULTS: The primary outcome was inhospital 28-day mortality by calendar month of admission, from March 2020 to January 2021. Unadjusted mortality was estimated, and Cox proportional hazard models were used to estimate adjusted mortality, controlling for age, sex, ethnicity, major comorbidities, social deprivation, geographic location, and operational strain (using bed occupancy as a proxy). Mortality fell to trough levels in June 2020 (ICU: 22.5% [95% CI, 18.2-27.4], high dependency unit: 8.0% [95% CI, 6.4-9.6]) but then subsequently increased up to January 2021: (ICU: 30.6% [95% CI, 29.0-32.2] and high dependency unit, 16.2% [95% CI, 15.3-17.1]). Comparing patients admitted during June-September 2020 with those admitted during December 2020-January 2021, the adjusted mortality was 59% (CI range, 39-82) higher in high dependency unit and 88% (CI range, 62-118) higher in ICU for the later period. This increased mortality was seen in all subgroups including those under 65. CONCLUSIONS: There was a marked deterioration in outcomes for patients admitted to critical care at the peak of the second wave of coronavirus disease 2019 in United Kingdom (December 2020-January 2021), compared with the post-first-wave period (June 2020-September 2020). The deterioration was independent of recorded patient characteristics and occupancy levels. Further research is required to determine to what extent this deterioration reflects the impact of the B117 variant of concern.

The association between mechanical ventilator compatible bed occupancy and mortality risk in intensive care patients with COVID-19: a national retrospective cohort study.

BACKGROUND: The literature paints a complex picture of the association between mortality risk and ICU strain. In this study, we sought to determine if there is an association between mortality risk in intensive care units (ICU) and occupancy of beds compatible with mechanical ventilation, as a proxy for strain. METHODS: A national retrospective observational cohort study of 89 English hospital trusts (i.e. groups of hospitals functioning as single operational units). Seven thousand one hundred thirty-three adults admitted to an ICU in England between 2 April and 1 December, 2020 (inclusive), with presumed or confirmed COVID-19, for whom data was submitted to the national surveillance programme and met study inclusion criteria. A Bayesian hierarchical approach was used to model the association between hospital trust level (mechanical ventilation compatible), bed occupancy, and in-hospital all-cause mortality. Results were adjusted for unit characteristics (pre-pandemic size), individual patient-level demographic characteristics (age, sex, ethnicity, deprivation index, time-to-ICU admission), and recorded chronic comorbidities (obesity, diabetes, respiratory disease, liver disease, heart disease, hypertension, immunosuppression, neurological disease, renal disease). RESULTS: One hundred thirty-five thousand six hundred patient days were observed, with a mortality rate of 19.4 per 1000 patient days. Adjusting for patient-level factors, mortality was higher for admissions during periods of high occupancy (> 85% occupancy versus the baseline of 45 to 85%) [OR 1.23 (95% posterior credible interval (PCI): 1.08 to 1.39)]. In contrast, mortality was decreased for admissions during periods of low occupancy (< 45% relative to the baseline) [OR 0.83 (95% PCI 0.75 to 0.94)]. CONCLUSION: Increasing occupancy of beds compatible with mechanical ventilation, a proxy for operational strain, is associated with a higher mortality risk for individuals admitted to ICU. Further research is required to establish if this is a causal relationship or whether it reflects strain on other operational factors such as staff. If causal, the result highlights the importance of strategies to keep ICU occupancy low to mitigate the impact of this type of resource saturation.

A national retrospective study of the association between serious operational problems and COVID-19 specific intensive care mortality risk.

OBJECTIVES: To describe the relationship between reported serious operational problems (SOPs), and mortality for patients with COVID-19 admitted to intensive care units (ICUs). DESIGN: English national retrospective cohort study. SETTING: 89 English hospital trusts (i.e. small groups of hospitals functioning as single operational units). PATIENTS: All adults with COVID-19 admitted to ICU between 2nd April and 1st December, 2020 (n = 6,737). INTERVENTIONS: N/A. MAIN OUTCOMES AND MEASURES: Hospital trusts routinely submit declarations of whether they have experienced 'serious operational problems' in the last 24 hours (e.g. due to staffing issues, adverse weather conditions, etc.). Bayesian hierarchical models were used to estimate the association between in-hospital mortality (binary outcome) and: 1) an indicator for whether a SOP occurred on the date of a patient's admission, and; 2) the proportion of the days in a patient's stay that had a SOP occur within their trust. These models were adjusted for individual demographic characteristics (age, sex, ethnicity), and recorded comorbidities. RESULTS: Serious operational problems (SOPs) were common; reported in 47 trusts (52.8%) and were present for 2,701 (of 21,716; 12.4%) trust days. Overall mortality was 37.7% (2,539 deaths). Admission during a period of SOPs was associated with a substantially increased mortality; adjusted odds ratio (OR) 1.34 (95% posterior credible interval (PCI): 1.07 to 1.68). Mortality was also associated with the proportion of a patient's admission duration that had concurrent SOPs; OR 1.47 (95% PCI: 1.10 to 1.96) for mortality where SOPs were present for 100% compared to 0% of the stay. CONCLUSION AND RELEVANCE: Serious operational problems at the trust-level are associated with a significant increase in mortality in patients with COVID-19 admitted to critical care. The link isn't necessarily causal, but this observation justifies further research to determine if a binary indicator might be a valid prognostic marker for deteriorating quality of care.

Type 2 Diabetes and COVID-19-Related Mortality in the Critical Care Setting: A National Cohort Study in England, March-July 2020.

OBJECTIVE: To describe the relationship between type 2 diabetes and all-cause mortality among adults with coronavirus disease 2019 (COVID-19) in the critical care setting. RESEARCH DESIGN AND METHODS: This was a nationwide retrospective cohort study in people admitted to hospital in England with COVID-19 requiring admission to a high dependency unit (HDU) or intensive care unit (ICU) between 1 March 2020 and 27 July 2020. Cox proportional hazards models were used to estimate 30-day in-hospital all-cause mortality associated with type 2 diabetes, with adjustment for age, sex, ethnicity, obesity, and other major comorbidities (chronic respiratory disease, asthma, chronic heart disease, hypertension, immunosuppression, chronic neurological disease, chronic renal disease, and chronic liver disease). RESULTS: A total of 19,256 COVID-19-related HDU and ICU admissions were included in the primary analysis, including 13,809 HDU (mean age 70 years) and 5,447 ICU (mean age 58 years) admissions. Of those admitted, 3,524 (18.3%) had type 2 diabetes and 5,077 (26.4%) died during the study period. Patients with type 2 diabetes were at increased risk of death (adjusted hazard ratio [aHR] 1.23 [95% CI 1.14, 1.32]), and this result was consistent in HDU and ICU subsets. The relative mortality risk associated with type 2 diabetes decreased with higher age (age 18-49 years aHR 1.50 [95% CI 1.05, 2.15], age 50-64 years 1.29 [1.10, 1.51], and age ≥65 years 1.18 [1.09, 1.29]; P value for age-type 2 diabetes interaction = 0.002). CONCLUSIONS: Type 2 diabetes may be an independent prognostic factor for survival in people with severe COVID-19 requiring critical care treatment, and in this setting the risk increase associated with type 2 diabetes is greatest in younger people.

Improving Survival of Critical Care Patients With Coronavirus Disease 2019 in England: A National Cohort Study, March to June 2020.

OBJECTIVES: To measure temporal trends in survival over time in people with severe coronavirus disease 2019 requiring critical care (high dependency unit or ICU) management, and to assess whether temporal variation in mortality was explained by changes in patient demographics and comorbidity burden over time. DESIGN: Retrospective observational cohort; based on data reported to the COVID-19 Hospitalisation in England Surveillance System. The primary outcome was in-hospital 30-day all-cause mortality. Unadjusted survival was estimated by calendar week of admission, and Cox proportional hazards models were used to estimate adjusted survival, controlling for age, sex, ethnicity, major comorbidities, and geographical region. SETTING: One hundred eight English critical care units. PATIENTS: All adult (18 yr +) coronavirus disease 2019 specific critical care admissions between March 1, 2020, and June 27, 2020. INTERVENTIONS: Not applicable. MEASUREMENTS AND MAIN RESULTS: Twenty-one thousand eighty-two critical care patients (high dependency unit n = 15,367; ICU n = 5,715) were included. Unadjusted survival at 30 days was lowest for people admitted in late March in both high dependency unit (71.6% survival) and ICU (58.0% survival). By the end of June, survival had improved to 92.7% in high dependency unit and 80.4% in ICU. Improvements in survival remained after adjustment for patient characteristics (age, sex, ethnicity, and major comorbidities) and geographical region. CONCLUSIONS: There has been a substantial improvement in survival amongst people admitted to critical care with coronavirus disease 2019 in England, with markedly higher survival rates in people admitted in May and June compared with those admitted in March and April. Our analysis suggests this improvement is not due to temporal changes in the age, sex, ethnicity, or major comorbidity burden of admitted patients.

Digital Health Management During and Beyond the COVID-19 Pandemic: Opportunities, Barriers, and Recommendations.

During the coronavirus disease (COVID-19) crisis, digital technologies have become a major route for accessing remote care. Therefore, the need to ensure that these tools are safe and effective has never been greater. We raise five calls to action to ensure the safety, availability, and long-term sustainability of these technologies: (1) due diligence: remove harmful health apps from app stores; (2) data insights: use relevant health data insights from high-quality digital tools to inform the greater response to COVID-19; (3) freely available resources: make high-quality digital health tools available without charge, where possible, and for as long as possible, especially to those who are most vulnerable; (4) digital transitioning: transform conventional offline mental health services to make them digitally available; and (5) population self-management: encourage governments and insurers to work with developers to look at how digital health management could be subsidized or funded. We believe this should be carried out at the population level, rather than at a prescription level.